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During the past decade, phase-transition phenomena in the random 3-satisfiabihty (3-SAT) prob- 
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3-SAT problem by the mean-field first-step replica-symmetry-broken cavity theory at the limit of 
temperature T ^ 0. The reweighting parameter y of the cavity theory is allowed to approach infin- 
ity together with the inverse temperature /3 with fixed ratio r — y/ f3. Focusing on the the system's 
space of satisfiable configurations, we carry out extensive population dynamics simulations using 
the technique of importance sampling and we obtain the entropy density s(r) and complexity E(r) 
of zero-energy clusters at different r values. We demonstrate that the population dynamics may 
reach different fixed points with different types of initial conditions. By knowing the trends of s(r) 
and S(r) with r, we can judge whether a certain type of initial condition is appropriate at a given 
r value. This work complements and confirms the results of several other very recent theoretical 
studies. 
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I. INTRODUCTION 

Critical behaviors in the random 3-satisfiability (3- 
SAT) problem were first reported by Kirkpatrick and 
Selman in 1994 Since then, physicists working in 
the field of spin glasses have done a lot of work on this 
important model system in theoretical computer science 
0,1^. Mean- field calculations were done to understand 
the nature of the satisfiability (SAT-UNSAT) transition 
111,11, H, to locate the SAT-UNSAT transition point 
a LZI, |8[ , and to analyze the performances of various al- 
gorithms f9|. Based on the first-step replica-symmetry- 
broken (IRSB) mean-field cavity theory of spin glasses 
[lOj . Mezard, Parisi, and Zecchina created a powerful 
message-passing algorithm, namely survey propagation 
(SP), to find satisfiable solutions to random 3-SAT for- 
mulas 0. The physical picture underlying the SP algo- 
rithm is that, when the density of constraints a of the 
system is close to the satisfiability threshold as , the solu- 
tion space of a random 3-SAT formula divides into many 
well-separated clusters. Mezard and co-workers also pre- 
dicted that the SAT-UNSAT transition for the random 
3-SAT problem occurs at Us = 4.2667 0,11. This thresh- 
old value lies within the rigorously known lower-bound 
3.52 [ll| and upper-bound 4.506 [ll for random 3-SAT, 
and the mean-field cavity SP solution is locally stable 
The predicted SAT-UNSAT transition point 
of as = 4.2667 is therefore conjectured to be exact. 

The message-passing SP algorithm corresponds to the 
temperature T = (i.e., f} = 1/T = +oo) limit of the 
IRSB mean-field cavity theory of finite-connectivity spin 
glasses [13, [iH. This IRSB cavity theory has an ad- 
justable reweighting parameter y. In Refs. 0,0,0], first 



the inverse temperature (3 is set to infinity, and then y is 
set to infinity. This means that the ratio ImiT^oy / (3 is 
equal to zero. On the other hand, it is now recognized 
that, to correctly characterize the equilibrium properties 
(as represented by the free-energy Gibbs measure) of a 
spin glass system, the reweighting parameter y is required 
to take an appropriate value that is dependent on (3. For 
a spin glass system with many-body interactions, there 
may exist a temperature range T^i > T > Tc within which 
the o ptim al value of the reweighting parameter y is equal 
to B |l6l . [itI . [Tsj . In the literature on structural glasses 
, Td and Tc are referred to as the dynamical and static 
transition temperature of the system, respectively. For 
the random 3-SAT problem with density of constraints 
a, if the corresponding static transition temperature is 
located at T'c(a) — 0, then the reweighting parameter y 
and the inverse temperature (3 should approach infinity 
with the same rate. In the present work, we investigate 
how the mean-field predictions on the ground-state prop- 
erties of the random 3-SAT problem depend on the ratio 
y/p. We generalize the cavity treatment of Refs. @, 0, Hi 
and study the statistical mechanics properties of the ran- 
dom 3-SAT problem in the limit /3 — > -l-oo and y +oo, 
with fixed ratio Il9l 
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Population dynamics simulations were performed based 
on a set of mean- field IRSB cavity equations, and for 
each value of a, the entropy density s(r) and complex- 
ity E(r) of the system as a function of the ratio r are 
estimated. The entropy density s(r) is a measure of the 
number of ground-energy configurations within one clus- 
ter of the configuration space, while the complexity S(r) 
is a measure of the total number of such ground-energy 
clusters. 

As the population dynamics simulations of this work 
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were running, we noticed that questions closely related 
to the issue we discuss here were investigated earlier in 
Ref. [ill in the context of the random 3-coloring problem 
and more recently in Refs. [ISj HH for random g-coloring 
and random if- SAT. While the main focus of Ref. [23| 
was on the limiting case of r = 1, at which the numer- 
ical complexity of the mean-field theory can be reduced 
to some extent, detailed discussions on general values of 
< r < 1 were presented in Refs. jll], |23]. The present 
paper confirms the physical picture given by Krzakala, 
Montanari, and co-workers [l^, [U, [23| on the solution 
space structure of random 3-SAT; it is complementary to 
these theoretical studies in three important ways. First, 
we introduce a different scheme of population dynam- 
ics with importance sampling (this scheme can be read- 
ily extended to finite temperatures); the numerical re- 
sults obtained from this scheme are in agreement with 
those reported in Ref. |22|| . Second, we demonstrate that 
the population dynamics may reach different fixed points 
from different initial conditions. Third, we find that dif- 
ferent initial conditions will lead to the same prediction 
on the properties of the dominating solution clusters of 
random 3-SAT. This last point is rather interesting and 
needs to be further studied. 

The main results of this paper are summarized here. 
When using the F-type initial condition as described in 
Sec. Ill C[ the population dynamics demonstrates that (i) 
at a = tts = 4.2667, S(r) decreases monotonically with r 
according to 'S,{r) — — 0.020r^ and s(r) increases mono- 
tonically with r; (ii) at a — 4.2, the complexity changes 
with r following S(r) = 0.0059 - 0.023r2 and s(r) still 
increases monotonically with r; (iii) at a = 4.0, both 
S(r) and s(r) have a discontinuity at r = 0. When us- 
ing the [/-type initial condition of Sec. Ill C[ we find that 
both S(r) and s(r) are not monotonic functions of r. At 
the value of r = 1, the complexity S(l) and entropy den- 
sity s(l) as a function of the constraint density a are also 
calculated by population dynamics simulations with both 
the i^-type and the J7-type initial condition. The numer- 
ical data are consistent with the conclusion of Ref. [l^l 
that, for a < 3.87 the solution space of the random 3-SAT 
problem forms a single cluster, while for 3.87 < a < as 
the solution space, although being nonergodic, is domi- 
nated by only a few (of order unity) solution clusters. 

The paper is organized asfoUows. Section [Til describes 
the mean-field cavity approach and the protocol of pop- 
ulation dynamics simulations. The simulation results are 
reported and analyzed in Sec. lIIII We conclude our work 
in Sec. IIVI and discuss possible future extensions. 



II. METHOD 

A. The factor-graph representation of the random 
3-SAT problem 

A 3-SAT formula contains TV Boolean variables and M 
constraints, each of which involves K — 3 variables. The 
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FIG. 1: The factor-graph representation 0, 0, [IJ for a ran- 
dom 3-SAT formula. mj_,b, rui-^a and Xb^i messages on 
the edges of the factor-graph (explained in the main text). 



degree of constrainedness of a random 3-SAT formula is 
characterized by the constraint density a = M/N. A 
3-SAT formula can be represented by a factor graph Q 
(see Fig.[T]) of TV variable nodes (circles fc, . . .) and M 
function nodes (squares a, b,c, . . .) [1, 0, IIS]- Each func- 
tion node a corresponds to a constraint; it is connected 
to K (= 3) variable nodes i ^ da (where da denotes the 
set of nearest neighbors of node a). Associated with each 
function node a is an energy Ea £ {0,2} of the form 

Ea^2ll'—^. (2) 

In Eq. ([2]), CTi = ±1 is the spin value of variable node i; 
J* = ±1 is the coupling between node i and node a. In 
the factor graph, the edge {i, a) is a soHd line if = 1 
and it is a dashed line if = —1. For a given 3-SAT for- 
mula, the factor graph (with all its coupling constants) is 
fixed, while the spin configuration a = {cti, (72, . . . , ctn} 
can change. The total energy E(a) of a given spin con- 
figuration is 

E{a) = J2Ea. (3) 

A variable node i of the factor graph Q is connected 
to ki function nodes a G di. the vertex degree ki may 
be different for different variable nodes. For a random 
3-SAT formula with 1, the distribution of ki is 

governed by the Poisson distribution of mean 3a, i.e., 
Prob(/ci = fc) = fsaik) = (3a)'=e~3"/A:!. One can also 
define the "cavity degree" ki^a of a variable node i with 
respect to an edge (a,i) as ki^a = \di\a\. ki^a is the 
number of nearest neighbors of node i when edge (a, i) is 
not considered. Obviously, fc^^a = ki — 1. A useful prop- 
erty of random graphs is that the distribution of ki^a 
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is also governed by the Poisson distribution of mean 3a. 
We will use this property in the mean-field population 
dynamics simulations as described in Sec. Ill CI 



B. The cavity equations at a general low 
temperature T 

At a sufficiently low temperature T, ergodicity of the 
whole configurational space A of the model Eq. ([3l breaks 
down, ft is then assumed in mean-field theories [a. [tI. ITo| 
that A is split into an exponential number of ergodic 
subspaces. Each of these subspaces Aq corresponds to a 
macroscopic state (macrostate a) of the system at tem- 
perature T. Based on the cavity approach of spin glasses 
[lOl [13] , the mean grand free-energy density of the ran- 
dom 3-SAT problem can be derived. As the derivation 
details are well documented in the literature 

0,ii3 (see 

also Refs. [11,111]), we shall directly list the final expres- 
sions and give only brief explanations. 

At the IRSB level of approximation, the total grand 
free energy of the random K-SAT system is 

Gg (/?; y)=J2AG,-{K-l)J2 AGa , (4) 

where AGi and AGa are, respectively, the grand free- 
energy increase caused by adding variable node i and 
function node a, with 



AG,; = - 
and 

AG, = - 



-log 
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(5) 
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In Eqs. (O and ©, m.i^a (the cavity magnetization) is 
the mean magnetization of vertex i within one macrostate 
a when the edge {a,i) is discarded, and Pi^ainT-i-^a) 
is the distribution of this cavity magnetization among 
all the macrostates of the system. Similarly, Xfc-^i = 
njgaf,\J(l — Jb''^j^b)/'2] is the directed message from 
function node b to variable node i in one macrostate, 
and Pb^i {xb^i ) is the distribution of this message among 
all the macrostates. AFi and AFa are, respectively, the 
free-energy increase of macrostate a due to the addition 
of variable node i and function node a, with 



AF, = 



log 



(-) 



n [1 - (1 - e-'^)xb^.] 
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AFa = - 
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(7) 
(8) 



In Eq. I?]), the Y[ and Y[ indicate that the multiplication 
is restricted to the neighbors 6 of i for which = — 1 and 
Jl = -1-1, respectively. 

On each edge {a,i) of factor graph Q, the probability 
distributions Pi^a and Pa^i are required to satisfy the 
variational condition that 



SGgiP;y) 5Gg{p-y) 



SPa- 



(9) 



. This variational condition is satisfied by the following two 
self-consistent equations on each directed edge a — > i and 
i — > a: 



Pa~.iiXa^t) = Yl [ <^mj^aPj^aimj^a)]Sixa^i~ 



and 



(1 - Jiruj^a) 



j^da\i 



j^da\i 



(10) 



n [/ Axb^^Pb^^{x.b^^)\e-y^^-'^5{m,^a " M {{xb^, : b e di\a}) 

n U dXb^^Pb^^{Xb^^)]e~y^^'-- 
bGdi\a 



(11) 



with M{{xb^i ■ b e di\a}) being the shorthand notation for 



M{{xb^, ■■ b e dt\a}) = 



(-) (+) 

n [l~{l~e-'^)xb^,]- n [l-{l-e-^P)xb^,] 
n [l-{l-e-^P)xb^,]+ n [1 - (1 - e-2^)x6^,] 

b^di\a b£di\a 



(12) 
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The free-energy increase AFi^a in Eq. pT|) is calculated completely contributed by entropy. 

by Eq. but with b G di being replaced by b S di\a p^^ ^^e benefit of later discussions, let us introduce two 
[i.e., discarding the effect of edge (z, a)] . ^^^^Yibt shorthand notations and Za, 



The T — > limit and population dynamics ^ ^ 

simulations Zi = Yli^ Xb^i) + Yli^ Xb-,i) , (13) 



bedi bedi 



Let us now consider the zero-temperature limit (i.e., Z ^ 1 TT ^ '^a'^^j^a) 

j3 — > -|-oo) of the cavity equations of the preceding sub- " ~ ii 2 ' 

section. We focus on the SAT phase of the random 3-SAT 
problem and assume the Hamiltonian Eq. ^ has at least 

one zero-energy ground state. In the SAT phase at the Then at /3 ^ co and fixed ratio r, the grand free-energy 

j3 +00 limit, the free energy of each macrostate is increases AG^ and AG^ can be reexpressed as 



2/AG, = -log[n(/ Axb-,^Pb-,^{Xb-,^))Q{Z,)e-'°^'^^'^] , (15) 

b&di 

yAGa = -l0g[l[{ f dm,^aP^^a{m^^a))Q{Za)e^'°^^^^^] ■ (16) 



In Eqs. © and 1^, e{x) = 1 if a; > and e(x) = if a; < 0. 

In the thermodynamic limit of graph size N —^ oo and M ^ oo (with a being finite), the grand free-energy density 
g{r) of the random 3-SAT system is expressed as 



ygir) = yAG^ ~ 2ayAGa , (17) 

where the overlines indicate averaging over all the possible local environments of the involved variable node i or 
function node a. The complexity S(r) and mean entropy density s(r) of the system are related to g{P]y) by (see, 
e.g., Ref. dl) 

S(r) - -yg{r) + r(^{l3AF,) - 2a{pAFa)) , (18) 



sir) = -i^{pAF)-2a{pAFa)j . (19) 
The mean free-energy increase of AFi and AFa as averaged over all the macrostate of the system is calculated through 

n [/dXb^,A^,(Xfc^.)]e(Z,)e'-i°g(^-) log(ZO 
iPAF,) = . , (20) 

n [IdXb^^Pb-.^{Xb^^)]eiZ,)er^°siZ.) 
bedi 

n [/ dm,^aP,^,(m,^„)]e(Za)e'-'°g(^»)log(Za) 
^^^^"^ " n [/ dm,^„P,^„(m,^,)] e{Z,)er^o,iZ.) ' ^^^^ 

I 



At a given value of constraint density a, we use pop- 
ulation dynamics [l3] to calculate the complexity E(r) 
and entropy density s(r) for the random 3-SAT problem. 
The iterative equation for the cavity magnetization 
distributions Pi^airrii^a) are implemented according to 
the following protocol of importance sampling. 

(1) . A total number of M sets are stored in the com- 
puter memory. Each set, which represents a proba- 



bility distribution Pi^a{iTii^a) of a cavity magne- 
tization, contains Ai double-precision values — 1 < 
rui^a £ 1- These M sets are independently initial- 
ized according to a certain type of initial condition 
(see below). 

(2) . To perform a single update to the stored population 
of distributions, the follow steps occur: 
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(i) A random integer n is generated according to 

the Poisson distribution fsain). 

(ii) 2n sets (denoted by 

Pj^^bt , Pfei^bi , . . . , Pj„^b„ , Pk„^b„ ) are 
randomly chosen with replacement from the 
stored N sets, and 3n coupling constants 
{ji^'^^,J^',Jl^} are generated, each of which 
is independently assigned a value +1 or — 1 
with probability one-half. 

(iii) 2n cavity magnetizations {rrij-^^bn'mki^bi), 
. . ., (rrij^^^bn, are sampled uniformly 
from these 2n sets, respectively, and n values 
Xbi^z = [(1- Jb^;TOj,^6,)(l-Jb;r7ifc,^b,)/4are 
calculated. 

(iv) Z, = n„,:J^^=-l(l - Xb.^r) + Ubr.Jl^=+li^ - 

Xbi-*i) is calculated and a new cavity mag- 
netization nii^a = [I\b,-P =-i(l ~ Xbi^i) - 
n6,:j^^=+i(l - Xb,^^)]/Z^ IS Calculated. 

(v) This new mi—,a value is accepted with proba- 
bihty proportional to 9(Zi) exp[r log(Zi)] by 
way of the Metropolis importance-sampling 
method [1^ and, if it is rejected, the old rrii^a 
value is retained. 

(vi) Repeat (iii)-(v) a number of M x C times 
and generate a new set Pi^a with A4 inde- 
pendent nii^a values (sampled with interval 
£). Obtain the value of yAd and {l3AFi) 
as expressed by Eqs. p3|) and (^0)1 using the 
sampled data of these A4 x C repeats. 

(vii) Replace a randomly chosen stored old set 
with the newly generated set Pi—,a- 

(3) . Repeat step (2) three times to obtain three proba- 

bility distributions Pi-tairrii^a), Pj^ain^j^a), and 
Pk^a{mk^a)- A total number oi M x C triples 
{mi^a,'mj^a,'mk->a) are then sampled uniformly. 
From these sampled data, yAGa and {(3AFa) as 
expressed by Eqs. pB]) and (pij) are calculated. 

(4) . Repeat steps (2) and (3) a number 71 x TV of times 

for the population dynamics to reach a steady state 
and another number 72 x A/" of times to collect val- 
ues of yAG„ yAGa, (/3AF,}, and (/3AF,). From 
these collected values, the grand free-energy density 
g{r), the complexity S(r), and the mean entropy 
density s(r) are calculated according to Eqs. ([TT]), 
(HH]), and (Uni), respectively. The standard devia- 
tions of the numerical results are estimated by the 
bootstrap method [l^l . 

The above-mentioned population dynamics procedure 
is quite time-consuming. The total simulation time is 
roughly proportional to MMC{Ti + T2). We have used 
different sets of parameter values to reach a balance be- 
tween high numerical precision and computation time. 
The data reported in the next section are obtained with 



the following set of parameters: N = 1000, M = 2000, 
C = 50, Ti = 500, and T2 = 1500 (with the exception 
that, in Fig. [51 the simulation results at a = 3.875 and 
3.9, which are close to the ergodicity transition point of 
the random 3-SAT system, are obtained with M = 2016 
and L = 100). At each pair of values (a,r), this set of 
parameters leads to satisfactory numerical precision with 
a tolerable simulation time of about ten days (through a 
present-day personal computer). If we use I\f = 2000 and 
= 4000 in the simulation, the mean values of the cal- 
culated thermodynamic densities will not change much, 
while their standard deviations can be reduced to about 
half the level of those reported in the next section. 

It is recognized by test runs that the results of the 
population dynamics can have a strong dependence on 
initial condition. The set of self-consistent equations pU)) 
and (jlip for the random 3-SAT problem may have more 
than one stable fixed point. To investigate this initial 
condition dependence, we use the following two major 
types of initial conditions to produce numerical data of 
the next section. 

F-type. The cavity magnetization distribution 
Pi^aifni-ta) at the beginning of the popu- 
lation dynamics is set to be Pi^ai'mi^a) = 
0.45 Sim.^a, 1) + 0.45 S{m,^a, -1) + 0.1 uim.^a), 
where u{x) is the uniform distribution over 
— 1 < a: < 1. This initial condition assumes that 
the spin value of a vertex i is frozen in most of the 
macrostates. 

U-type. The initial cavity magnetization distribution 
Pi^airrii^a) is set to be Pi^aimi-^a) = u{mt^a) 
with — 1 < rrii^a < 1- This condition assumes that 
initially the spin value of a vertex is unfrozen in all 
the macrostates. 

The plausibility of each of these two initial conditions 
will be judged by its predictions. 

III. RESULTS 

A. Population dynamics at a = 4.2 

At a = 4.2, the complexity I](r) and the mean entropy 
density s(r) of the random 3-SAT are shown in Fig.[2]for 
< r < 0.75. Under the F-type initial condition, the ob- 
tained complexity values can be fitted by I](r) = a — br^ 
with a = 0.0059 ± 0.0002 and b = 0.023 ± 0.001, while 
the mean entropy density s{r) increases monotonically 
from s(0) = 0.060 ± 0.001 to s(0.75) = 0.097 ± 0.001. 
These results appear to be quite reasonable: (i) Accord- 
ing to Refs. [Us m, as r increases, the complexity 
should decrease and the mean entropy density should in- 
crease; (ii) the value of S(0) agrees with the prediction 
of the SP algorithm Q, which gives i;(0) = 0.00599; 
(iii) S(l) is negative, in agreement with Ref. The 
mean-field theory suggests that the solution space of a 
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FIG. 2: (Color Online) The complexity (a) and mean entropy 
density (b) for the random 3-SAT problem with constraint 
density a = 4.2. Black circles and red square are, respectively, 
simulation results obtained using the _F-type and the 17-type 
initial condition. The blue dashed line in (a) is a fitting to 
the circular points with E(r) = 0.0059(2) - 0.023(1) 



typical long random 3-SAT formula with constraint den- 
sity a = 4.2 is dominated by a few clusters of entropy 
density s « s(0.5) = 0.088 ± 0.001 [with S(0.5) « 0], al- 
though clusters of lower entropy density s = s(0) ~ 0.060 
are most abundant in the solution space. These two en- 
tropy density values are in agreement with the results of 
Ref. ^2^. 

When r > 0.4, the complexity and mean entropy den- 
sity values reported by the population dynamics with 
the t/-type initial condition are in agreement with those 
obtained with the F-type initial condition. For r > 
0.4, the mean-field population dynamics is insensitive 
to initial conditions. However, under the [/-type ini- 
tial condition the complexity ^(r) increases with r and 
the mean entropy density s(r) decreases with r when 
r G [0,0.4). This behavior is unphysical, because the 
mean entropy density s(r) should be an increasing func- 
tion of r [18(1 . Therefore, under the [/-type initial con- 
dition, the parameter r should not be set to values 



lower than 0.4. Under the [/-type initial condition, the 
fixed point of the population dynamics at r = corre- 
sponds to the replica-symmetric solution of the SP al- 
gorithm [?[. This replica-symmetric solution is always 
stable in the mean- field theory of Ref. 0, as entropic 
effects are completely neglected. When the entropy of 
each zero-energy macrostate is properly considered in the 
mean-field theory, the present paper indicates that this 
replica-symmetric solution is no longer stable (see also 
Refs. [2^,^]). To get physically meaningful results for 
< r < 0.4 under the [/-type initial condition, it is 
necessary to assume further organization of the solution 
space of the random 3-SAT problem (splitting of a cluster 
of solutions into many subclusters of solutions). Imple- 
menting this higher-order hierarchical structure into the 
population dynamics is conceptually simple, but the al- 
gorithm will be extremely demanding on computer time 
and memory space. 



B. Population dynamics at a = 4.2667 

The SAT-UNSAT transition of the random 3-SAT 
problem is predicted to occur at a = 4.2667 0, Hi- At 
this density of constraints. Fig. [3] shows how the com- 
plexity and entropy density change with the ratio r. Un- 
der the i<"-type initial condition, the complexity decreases 
with r according to i;(r) = -br^ with b = 0.020 ± 0.001; 
and consistently, the entropy density s{r) increases with 
r monotonically. The present work, therefore, further 
confirms that the satisfiability transition of the random 
3-SAT takes place at a = 4.2667: including the entropic 
effect into the mean-field theory does not change the 
predicted location of the SAT-UNSAT transition. At 
this transition point, a typical random 3-SAT formula 
of length N still has an exponential number exp[A^s(0)] 
of satisfiable solutions, with s(0) = 0.058 ± 0.001. But 
it is extremely difficult for a local or global algorithm to 
find one such solution. 

As in the case of a = 4.2, if the [/-type initial condition 
is applied, both the calculated complexity S(r) and mean 
entropy density s(r) do not change monotonically with 
r. Figure [3] indicates that the results from the [/-type 
initial condition are valid only for r > 0.25. For < r < 
0.2, as the increasing trend of the complexity S(r) and 
the decreasing trend of the entropy density s(r) are not 
physically meaningful, the positivity of S(r) cannot be 
taken as evidence that the random 3-SAT is still in the 
SAT phase at a = 4.2667. 



C. Population dynamics at a = 4.0 

For a = 4.2667 and 4.2, the complexity S(r) calcu- 
lated with the -F-type initial condition reaches maximum 
at r = and it has the form S(r) = a — br^ when 
r e [0, 1). However, Fig. 0] demonstrates that a dif- 
ferent situation occurs for a = 4.0. At this density of 
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FIG. 3: (Color Online) The complexity (a) and mean entropy 
density (b) for the random 3-SAT problem with constraint 
density a = 4.2667. Black circles and red square are, respec- 
tively, simulation results obtained using the F-type and the 
?7-type initial condition. The blue dashed line in (a) is a fit- 
ting to the circular points with E(r) = —0.020(1) r'^, while 
the black dotted line marks E(r) = 0. 



constraints, the population dynamics with r = and 
the -F-type initial condition reports a complexity value 
E(0) = 0.0217 ± 0.0006 (agreeing with the prediction 
of the SP algorithm Q) and a mean entropy density 
value s(0) = 0.069 ± 0.003. But as the ratio r is set to 
slightly positive values, the complexity suddenly drops 
to S(r) « while the mean entropy density jumps to 
s(r) « 0.125. As r increases further, both I](r) and s{r) 
keep almost constant until r is close to unity. For r > 0.8, 
E(r) and s(r) have, respectively, a decreasing and an in- 
creasing trend. The discontinuity at r = for both S(r) 
and s(r) was totally unexpected (we have performed pop- 
ulation dynamics simulations with different _F-type ini- 
tial conditions to rule out the possibility of numerical 
artifact). Similar discontinuity was also observed in the 
g-coloring problem |21||. If we look at the steady-state 
cavity magnetization distributions Pi^aimi—^a), we find 
that they are far from being in the form of a (5-function 
in the whole range of < r < 1. This later observation 



FIG. 4; (Color Online) The complexity and mean entropy 
density (inset) for the random 3-SAT problem with constraint 
density a = 4.0. Black circles are simulation results obtained 
using the F-type initial condition. The blue dashed line marks 
E(r) = 0. 



confirms that at a = 4.0, the ergodicity property of the 
solution space of the random 3-SAT is indeed violated. 
Figure [4] indicates that at a = 4.0, the solution space of 
the random 3-SAT problem is organized far more com- 
plex than what has been assumed in the mean-field the- 
ory. This point should be investigated more thoroughly. 

For the limiting case of r = 0, it has already been 
shown that the mean-field solution at the first-step 
replica-symmetry-broken level is unstable toward the 
full-step replica-symmetry-broken level [1, [l^] for a < 
4.153. The different behaviors demonstrated in Figs. [2l 
[31 and H for a = 4.2^.2667, and 4.0 confirm the ear- 
lier stability analysis [1, [l^ and further suggest that, if 
the IRSB mean-field solution is unstable at r = 0, it 
will be unstable when r is positive but less than a cer- 
tain threshold value rth- This threshold value may be 
smaller or larger than unity (for a = 4.0, it appears that 
rth ~ 0.8). 



D. Population dynamics at r = 1 

Now let us fix 7' = 1 and study how the complexity 
S(l) and mean entropy density s(l) change with the con- 
straint density a. Using an elegant tree reconstruction 
technique, Montanari and co-authors [l^l found that, for 
the random 3-SAT problem, S(l) changes from being ex- 
actly zero to being negative at a « 3.87. The alternative 
population dynamics approach of the present paper re- 
ports consistent results (see Fig. [S]). For a = 3.8 and 
3.85, we have checked that the steady-state distributions 
Pi^a(mi^a) of cavity magnetizations are all (^-functions 
(ergodicity property of the solution space is not violated) . 
For a > 3.875, simulations with both the F-type and 
U -type initial condition give negative values for the com- 
plexity S(l). At a very close to the ergodicity transi- 
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FIG. 5: The complexity E(r — 1) and the mean entropy den- 
sity s(r = 1) for the random 3-SAT problem with constraint 
density a. Black circles and red squares correspond to the F- 
type and the t/-type initial condition, respectively. The blue 
dotted line marks S = 0. 



tion point of 3.87, we have also observed that the pop- 
ulation dynamics simulation needs a much longer time 
to reach steady state. This behaviors is very probably 
caused by the divergence of relaxation times of the pop- 
ulation dynamics at the vicinity of the ergodicity transi- 
tion (a w 3.87). Such a critical slowing-down was inves- 
tigated analytically and numerically in Ref . psj . 

When a > 3.87, very probably most of the satisfying 
solutions of a random 3-SAT formula can be grouped into 
one of a subexponential number of clusters of solutions 
[Tsl [20| . It will then be very difficult to prove mathemat- 
ically the clustering of solutions following the method of 
Ref. M. 



IV. CONCLUSION AND DISCUSSION 

In this paper, we studied a spin glass model of the 
random 3-SAT problem at the temperature T limit 
by the mean-field first-step replica-symmetry-breaking 
(IRSB) cavity method. The reweighting parameter y 
(corresponding to the level of macrostates) and the in- 
verse temperature /3 were allowed to approach infinity 
with fixed ratio r — y/(3. The complexity and mean en- 
tropy density of the random 3-SAT are calculated as a 
function of r by population dynamics simulations. The 
sensitivity to initial conditions of the simulation results 
was investigated by initializing the cavity magnetization 
distributions in two different way (see Sec. Ill C| ). 

When the F-type initial condition is used, at a = 4.2 
the complexity S(r) decreases monotonically with r and 
becomes negative when r exceeds 0.5; the mean entropy 
density s(r) increases monotonically with r. The most 
abundant clusters of solutions of the random 3-SAT sys- 
tem correspond to r = and have mean entropy density 
s(0) ~ 0.060, but the (few) dominating clusters of solu- 
tions correspond to r « 0.5 and have mean entropy den- 



sity s(0.5) « 0.088. The complexity S(r = 0) decreases 
continuously with a and reaches zero at a = 4.2667, 
where the random 3-SAT experiences a SAT-UNSAT 
transition. At this critical constraint density, the solu- 
tion space of the random 3-SAT still has a positive mean 
entropy density s(0) « 0.058. 

When the J7-type initial condition is applied, the com- 
plexity S(r) and mean entropy density s(r) are both non- 
monotonic functions of r. At a = 4.2, the population 
dynamics algorithm reported a zero complexity value at 
r — Q. As r becomes positive, S(r) first increases with r, 
reaches a maximal value at r « 0.4, and then decreases 
with r. The mean entropy density s(r) has a reverse 
trend. The non-monotonic behaviors of S(r) and s{r) 
indicate that, for the ?7-type initial condition the pop- 
ulation dynamics will not report physically meaningful 
results if r is close to zero. At a = 4.0, if the parame- 
ter r is set close to zero, even the population dynamics 
with the F-type initial condition will fail to get plausible 
results. 

At r 1, the complexity and mean entropy density as 
a function of constraint density a were also investigated 
by population dynamics. For a = 3.85 or lower, ergodic- 
ity of the solution space of the random 3-SAT is unbro- 
ken and the complexity is exactly zero. For a = 3.875 or 
higher, the population dynamics with both the F-type 
and the ?7-type initial condition predicted negative val- 
ues for S(l). The zero-energy configuration space of the 
random 3-SAT problem clusters into many subspaces for 
a > 3.875, but only subexponential clusters are dominat- 
ing the configuration space, in agreement with Ref. ^20j. 

This paper focused on the zero-energy configurational 
space of the random 3-SAT problem. When the ground- 
state energy of the system becomes positive, the T — » 
limit formulas in Sec. Ill CI need to be revised. Most im- 
portantly, in a given macrostate a cavity magnetization 
rui^a may take one of the following three possible forms: 



1 



-2/3 



-2/3 



where — 1 < nrq^„ < 1, > 0, and m 



(22) 



In 



the present paper, we have simply set ti^^ = ~ 
without affecting the results of population dynamics, 
but for systems with positive ground-state energies, the 
more general formula should be used. Even if the ground- 
state energy of the system is zero, Eq. ([^ should be 
used if one wants to study the properties of metastable 
macrostates (with positive minimal energies) or the low- 
temperature properties of the system. We will return to 
this point in a later publication. 

As Refs. [20l . [22j and the present paper demonstrate, 
the zero-energy configuration space of the random 3-SAT 
problem is divided into clusters of different sizes. For the 
random 3-SAT problem, will the minimal-energy config- 
urations with a given positive energy value E also be 
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split into clusters of different entropies S? To detect 
such a possibility, a natural extension is to introduce two 
reweighting parameters (say y and r) for both energy and 
entropy, and to reweight each minimal-energy cluster a 
by a factor exp{—yE + rS). Together with Krzakala and 
Zdeborova, we are working on this point for the random 
3-SAT problem and the g-coloring problem. 

Although physicists believe that the solutions of a large 
random 3-SAT formula are organized into well separated 
subspaces, clustering of random if-SAT solutions has 
been rigorously proven only for K > 8 [29]. Recently, 
there has been a lot of simulation work on this impor- 
tant issue (e.g., [S^, [3l|), but a lot of work still remains 
to be done to fully understand the energy landscape of 



the random 3-SAT problem. 
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